Attention allocation for decision making queues

We consider the optimal servicing of a queue with sigmoid server performance. There are various systems with sigmoid server performance including systems involving human decision making, visual perception, human-machine communication and advertising response. The tasks arrive at the server according to some stochastic process. Each task has a deadline that is incorporated as a latency penalty. We investigate the trade-off between the reward obtained by processing the current task and the penalty incurred due to the tasks waiting in the queue. We study this optimization problem in a Markov decision process (MDP) framework. We characterize the properties of the optimal policy for the MDP and show that the optimal policy may drop some tasks, that is, may not process a task at all. We determine an approximate solution to the MDP using certainty-equivalent receding horizon optimization framework and determine performance bounds on the proposed receding horizon policy. We also suggest guidelines for the design of such queues.